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A  Bayesian  Approach  to  some  Missing  Value  Problems 

# 

in  ANOVA  and  Contingency  Tables 
by 

Daniel  Bloch 


1.  Introduction 

Non-Bayesian  procedures  for  dealing  with  missing  values 
in  ANOVA  and  contingency  tables  are  well-known  (see  e.g.  [3]  and  [6]). 

In  this  paper  we  derive  the  appropriate  Bayesian  procedures  for  a 
randomized  block  design  and  for  an  r  x  c  contingency  table.  The  posterior 
distributions  for  the  missing  observation  are  also  given.  This  paper 
shows  that  the  classical  non-Bayesian  procedures  do  have  a  simple  and 
natural  Bayesian  interpretation. 

2.  Missing  values  in  a  randomized  block  design. 

Suppose  that  the  observations  in  a  b -block  x  t- treatment 
randomized  block  design  are  lnconplete  because  y^  Is  missing.  Our  model 
for  the  available  observations  is 


We 


(2.1) 

y±j  “  M>  +  €^j  ,  ( 1 ,  j)  ^  (l  *  l)>  i^L, . . .  ,b  ;  ...,t. 

b  t 

assume  that  ^  p^  ■  0,  ^  -  0,  the  are  independent  and 

i-1  J-OL 


2 

normally  distributed  with  mean  zero  and  variance  o'  .  (p,  is  the  over-all 

mean  effect,  (j««l,  ...,t)  is  the  treatment  effect,  and  Pi(i-1, . . .  ,b) 
is  the  i^*1  block  effect). 
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We  now  make  the  usual  "non -Informative"  assumption  that  the 

Joint  prior  distribution  of  (p,,  a,  t^'s,  f^'®)  is  proportional  to  l/a. 

From  the  resulting  posterior  distribution,  u,  o  and  the  ^'s  may  be 

integrated  out  to  obtain  the  distribution  of  most  interest — the  posterior 

distribution  of  the  Tj's.  The  lack  of  balance  caused  by  the  missing 

observation  makes  this  an  awkward  calculation.  If  however  we 

introduce  y11  as  a  random  variable,  this  balance  is  restored.  Since 

y^  -  p.  +  +  e^,  the  density  of  y^  given  p,  +  is  normal 

2 

wit.,  mean  p,  +  and  variance  a  .  Die  Joint  posterior  density 

of  (y^,  pi,  a,  Tj 's,  Pi's)  is  therefore  proportional  to  the  product  of  the 
conditional  normal  density  of  y^  and  the  lib  .ihood  of  the  available 
observations  and  o  Hence 


1  Likelihood  functi 

Post  (u>o,t  's,P1's,yi;L)  «— ^  expf—  for  the  available 


b  t 


Likelihood  function 
for  the  aval] 
observations 

(2.2) 


W5  exp  fe  [II  (>rij-‘1"Tj'Pi)2  3  }  • 


i-1  j-1 


Die  exponent  can  be  written  as 

b  t  t  b 

I  I  (yij-|‘-Tj-pi)2  ■  tbGr->*>+b  I  (yj-y-Tj>2  +  *  I  (^i-y-ei)2  +  s2< 

1-1  J-l  J«CL  i-1 


(2.3) 


*> 

■  J  ■ 


where 


b  t 


s2  •  I  1  -  *i.  -  *. }  * 

i-0.  J^l 

Substituting  (2.3)  into  (2.2)  we  have  that 


Post  (a,  r  y^)  “»  {rT  [s2  +  b  1  (y.3  -  *  ■  Tj)2]} 

a  zcr  ._s 


J-l 


(2-4) 


(b-l)-d^mensional 

space 
with 


1*0. 


sp<lce  I  ft-3 


1-0. 


t 

*  "tCT+r  “®  tr  [s2  +  b  I  (y.j  -  * 

0  ®  3< 


Denote  the  missing  observation,  y^,  by  x. 

I*t  ?•  -  y  -  £  ,  y|.  -  y.±.  -  £  ,  y^  -  y.i  -  £  • 


-  It  - 


It  is  easily  verified  that 


b  I  (y.j  -  ?  -  Tj)2  -  *  -  atfy’-y;!  +  tx) 

a-*  tb 


+  b  Z  (y.j_y:,"Tj^2+  b(y'i  "y'“Ti)' 
J-2 


and 


b  t 


s2-  I 

iJ.  j-a 


(2.5) 


+  2x(y'-y±,  -  y]x)  +  (y1  -  y{.-  y^)2  , 


(2.6) 


where  "/"  means  y^  and  y#^  are  to  be  replaced  by  y^.  and  y ^ 
respectively  and  the  summation  does  not  incluse  (i,j)  -  (l,l),  i.e.. 


b  t  , 
\  • 


b  t 

i  r  (yij  -  yi-  -  y.j  -y,>2  - 1 1  <yu  ■  ?f-  y.3 ♦ y,> 


-.^2 


i-1  J-l 


i-2  J-2 


1  <yu  -  yi-  ■  y.j  *y’)2  +  I  <yij-  yi.-y-’i +yl)2 


.J-2 


i-2 
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Prom  (2.5)  and  (2.6)  we  have  that  the  coefficients  of  the  x  and  2x  terms 
t 

from  S2  +  b  ^  (y#j  -  y  -  t^)2  are  (^)  and  -  (y£.  +  t1)2  respectively. 
J^L 


Since 


♦  v-m [*-^-]2- 


we  have  that 


Post  (ojTj's)oe  exp  [  i  G±fy±'-y.fv 'f  +  (y‘  -  /{.-y^)2 

a  ^  i,J 

(2.7) 

i  o  _  _  2  cr  .+0**  -1 1 

+  bI j-y'‘TP  •rt,(y:1-y,-Ti)  -  t-i  ]  1. 
J-e 


If  is  easily  shown  that 


_  _  _  2  -  -  2 

(y'-y^.-yli)  +  b(y.,1-y*-T1) - ~ —  - 


\  w  /  LI  bt-b-t  J  bt-b-t  x  1 


I 


Hence 


Post  (t  's  |  available  data) «  — = - i — 

J  r  C  +(r>a)' 


tb-b- 


vhere 


C  +(T*,a)'M(iF-a)] 


-• l t3"0> 


J-a 


(2.8) 


Mt-lJfr.'j-y'J+t  y£.  _  _  _  *' 

a"  V - bt^t - »  y.2 -y’J 

T  "  ( V **,,Tt)  , 


M  - 

txt 


and 


bt-b-t 
t-1 


O  \  ' 


/ 


t.j 


The  posterior  density,  (2.8),  is  constant  where  (T.-a)'  M(i-a)  is  constant. 
The  surfaces  ^T-a)'  M(T-a)  «  c  are  ellipsoids  in  the  (t-1 ) -dimensional 
Tj  ■  0.  Die  density  decreases  as  the  distance  from  the 


spare  with 


l 


j-1 

center  of  the  ellipsoids  Increases.  Hence  a  confidence  region  is  an 
ellipsoid.  The  same  argument  as  used  in  proving  theorem  6.4,1  in  [5] 


shows  that  if 
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Post  (0  |  available  observations )  « 


[(t-i)  +  [(t-i)<b-i)-i]0 


and  hence 


tb-b-1 


(2-9) 


;r-aVM(T-a)/(t-l) 

C2/(t-l)(b-l)-l 


?t-l,  (t-l)(b-l)-l. 


(2.10) 


An  a- level  test  of  the  null  hypothesis  that  there  are  no  treatment  effects 
lc  therefore  to  declare  the  results  significant  if 


aW(t-l)  -  (l-a) 

02/(t-l)(b-l)-i 

If  no  observations  are  missing,  then  it  can  be  verified  that  the  posterior 

(t-aJ'MT^Xt-l) 

distribution  of  ,  -  . .  is  F.  ,  w.  ,  \ 

1  s2/(t-i)(b-i)  t-Mt-iNb-1) 


where  -  (y>]L  -  y,  y>2  -  y,...,  y  -y) 


and  ■ 
txt 


\ 

0  b 


1Mb  is  the  same  as  the  sailing  result.  Hie  effect  of  the  missing 
observation  is  to  decrease  the  error  degrees  of  freedom  by  unity.  In  the 
analysis  a^,  and  B2  should  be  replaced  by  a,  M  and  C2  respectively. 
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We  have  ftrom  (2.4)  that 


2*i  pcLT%  « •  #cLTi 


t. 


R>st  u,o) «  J-  *t-l-  *  {fl  [b  £  (y,  j-7- 

J*d- 


V‘ 


( t -1 )  -  (Omens  ional 
spare  with  ^Tj-O 


(2.11) 


i  r-s' 

*  ~  tb^b~+2 


Hence,  using  the  expression  for  S  given  by  (2.6), 


*>at  Cx»a)«  exp^|  [  ^  (y^.^.j -&Y 


(2.12) 


+  x2  (t-1)(-b-~1l  +  2x(y»-y'  -y»  ) 
tb 


+  (y' 


+  tb 


(yj .-♦y.’-.-y' )tb-2  > 

(tri)Tb-D  "J  J  i 


upon  completing  the  square  with  respect  to  x.  Therefore  the  posterior 
distribution  for  the  missing  observation  x  is  given  by 
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Post  (x)  CC 


tb+b-t+1 


[ 


2  (t-i)(b-i)  .  ’  )tb\2n 


(2.13) 


D  + 


tb 


( 


(t-D(b-i) 


■)] 


/ 


where  D2  -  Y  (y,,-  y  -  y.  +  y '  )2  -  (y'-yj.-y’.  )2 

1J  3  1  (t-l)(b.l)  1  -1 


Notice  that 


(yJ  +  y'.  -  y '  )tb  bB+tT-b  ,  ..  .  »  .. 

1  •!  B _  yates  estimator  for  the 

(t-l)(b-l)  (t-l)(b-l)  missing  observation  x, 


(2.14) 


where  G  =  t  b  y'«*  Grand  total  of  the  available  observations. 

Bo  t  yJ  o  total  of  the  remaining  nnita  in  the  block  where  the 
missing  vnit  appears, 

Toby'  ■  total  of  the  yields  of  this  treatment  in  the  other 
blocks 

Prom  (2.13 )  we  have  that 


Post  (x) 


[l  +  ^{x)..j2j  (t-l)^b-l)_ 


(2.15) 


where  p,(x)  is  the  Yates  estimator  for  x  and  is  given  by  (2.14)  . 
If  we  let 


-  10  - 


then 

Post  (y)«  ~~ ■  n — -  ,i.e.  (2.l6) 

ri+  _2_] 

L  tb-b-t  J  2 

▼  is  distributed  as  a  student-t  random  variable  with  (tb-b-t)  =  (t-l)(b-l)-l 
degrees  of  freedom.  It  is  interesting  to  note  that  if  x  is  replaced 
by  u(x)  (2-6),  then  S  is  identical  to  D  .  Inferences  about  the 
missing  observation  x  should  be  made  by  referring  to  (2.16) 

3-  Missing  values  in  contingency  tables. 

Let  n^  be  the  cell  frequency  in  the  (ij)  cell  in  an  r  X  c 
contingency  table.  Let  be  the  probability  that  an  observation  lies 

in  the  (ij)^  cell.  If  no  cell  frequencies  are  missing,  then  under  the 
null  hypothesis  of  no  association  between  rows  and  columns 


PiJ  "5i  qJ  '  1  "  c' 


(3-D 


where  the  E‘  ’s  (q.'s)  are  the  probabilities  of  an  observation  falling 
1  J  r  c 

into  the  i^1  row  (j^*1  column),  ^  ^ 

i«sl  JnL 

If  is  missing,  then  under  the  null  hypothesis  the  Joint 

distribution  of  the  available  n^j's  *s  Siven  by  an  (rc-l)  -  nomial 
distribution  with  cell  px-obabilities 


P 


U 


riqj 


i  ,r;  J«4,...,  c;  (i,j)  f  (l,l) 


(3.2) 


-  n  - 


Let  N  be  the  total  of  available  frequencies. 

The  likelihood  function  for  the  available  frequencies  is 


l  .  w  pa l.)d^ 

(i,j)AlA)  l-P^ 


(3.3) 


Let  the  prior  distribution  for  the  missing  frequency  p(ry^)  say, 
be  given  by  the  negative  binomial  distribution 


p(V)  "  (  "  ^  )  (pi  «i> 


N 


^  (1-P,qj“,  “ll  ■  O’1’2’  -- 


(3-4) 


lfce  choice  of  (3-4)  for  the  prior  distribution  of  hj^  can  be  motivated 

by  noting  that  if  ry.  were  not  missing,  then  the  marginal  probability 

ry,  N 

of  observing  in  the  first  cell  would  equal  ^  n^J(p^q^)  (l-p^q^)  . 


We 


replace  (*^n)  by  (*““■  )  bo  that  £  pOfc)  -  1  . 
^  ”u-° 


If  the  prior  distribution  of  (p^,  •  •  •  >Pr>qj/  •  *  •  ,qc)  is  proportional  to 


a. 


/  V  tu  .  . 

'  fr  '  (p^j)  '  then  the  P°s-terior  ilistribution  of  the  pys,  qj's 

(iyj)-(lyl) 


and  iy  is  proportional  to 
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(»-.*>’  w  (t &)"“  "*-1 

(i,j)/(l,l)  (i,j)-(l,l) 


(3-5) 


In  the  derivation  below  we  take  all  equal  to  zero.  Non- zero 
values  can  usually  be  introduced  at  the  end.  Ffom  (3*5)  we  than 
have  that  the  Joint  posterior  distribution  of  the  'b  and  q^'s  is 
given  by 


root  (p^s. 


V’ 


-I 

“n"0 


^pi9i^nn^1”piqi^ 

(i,j),*(l,l) 


cc 


(r*c) 

(i,J),i(l,l) 


(3.6) 


Jeffreys  (see  [4])  showed  that  the  Joint  posterior  density  (3*6)  can, 
as  N  —  <*>,  be  approximated  by  the  normal  density  with  exponent  -  ^ 
where 


(r,c) 

**.  l 

(t.jWi.D 


(3-7) 


X2  is  an  approximation  to  the  likelihood  ratio  statistic 


C  - 


( r>c )  »  -i 

•2  i"ij 


(3-8) 


The  exact  posterior  distribution  of  C  is  obtainable  by  using  the 
methods  of  Watson  [7]  • 

The  quantity  X2  has  the  distribution  as  N  -*  ®. 

Hence,  using  theorem  7-5*l  in  [5l> 


(r,c) 

?-  1 

(i,j)Al,l) 


(v*  pij ) 


* ' 

PU 


(3.9) 


has  the  \  distribution  as  N  -♦  ®.  The 

(rc-2)-(r+c-2)»(r-l)(c-l)-l 


pjj's  axe  the  maximum  likelihood  estimates  for  the  p^'s  assuming  that 
the  null  hypothesis  of  no  association  between  rows  and  columns  is  true. 
Watson  [6]  showed  that  the  M.L.  estimates  of  the  Pj's,  q^'s  and 
are  given  by 


V°ll  £  R1  dm  O  ) 

>  Pi  "  — —  ,  (i  2,  ,  r) 

N+nll  N+fill 


V5!! 

N+“n 


N+] 


>  (j  m  2,  .  .  .  fc) 


*11 


(3.10) 


-  lb  - 


where  (i«&,  ...,r) 
column  totals .  From 


and  C  (j^l, . .  •  ,c)  are  the  existing  row  and 
J 

(3.10)  we  find  that  the  P£j's  are  given  by 


V.i 

N(N-C^) 


( J“2> •  •  •  ;  c ) 


PJ,  -  — ,  (i-fi,...,r)  (3.11) 

11  N(N-RX) 


r1cj(n-vc1> 

nCn-i^JCn-^) 


( i*^;  •  •  •  » r  .*2;  •  <  •  j  c  ) 


Hie  test  given  by  (3-9)  is,  operationally,  the  same  as  the  sampling  ■ 
result . 

The  Joint  distribution  of  (p^, . . .  ,pr  ;q^, . . .  ,qc  jn.^)  can  be 
rewritten  as 


Post 


■>V<,’nn) 


/N+nn"1\ 

l  nn  >1 


V°n 


:i+nll  r 
rr 

i*=2 


c 

TT 

J-2 


(3.12) 


From  the  normalizing  constant  of  the  Dirichlet  distribution 


we  know  that 
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4. 


p-, 


r  Ri  ,  r 

TT  P,  dp  ...d  P  »  TT 

1=2  x  1-2  (N+ajj+r-1)! 


where  -  jail  in  (0,1),  ^  PA  -  l} 


1=0. 


and 


l 


Cl+nTL  c  _  CJ.  .  c  Cj!(nll'fCl): 


tt  q.  ‘W.*  .da  -tt 

J  a  C  J-2  (N+n^+C-1)! 


J-2 


where  S2  -  jail  qj  in  (0,l),  ^  qj  -  1  } 


i«d 


Therefore 


Post 


(nn> 


“n 


t 


(N+n^+r-i)*  (N+nn+c-l)  » 


V 


0,1, 


We  use  BarneB'(see  [2])  asymptotic  series  for  log  r(x  +  h), 


—  i  r  (-l)PB_.,(h) 

log  r(x+h)^log  72tt  +  (x+h-  £)  log  x-x-  )  - - - * — —  +  R  .  (x 


P-l 


P  (p+l)  X 


(3.13) 


(3.14) 

), 
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where  the  Bp  (h)'s  are  the  Bernouille  polynomial  and  R^^x)  “  0(x"^n^'L^), 
to  obtain  the  large  sample  distribution  for  the  missing  frequency  a^  . 

We  find  that  log  Post  (a  .  )  is  proportional  to 


|)l°6(fi11+R1)+(nn+C1+  IjlogC^^J-CN+n^+r+c-  ^logtN+a^) 


m 


-log  r(o11+l)+(H-C1-R1-ilu)+^  {— Ifil 


P«M 


*  p(p+l)(w+fi11>1? 


(3.15) 


Bp+1  ^nll"”ll+r  ^+Bp+l  ^nll"“ll+G  ^ ”Bp+l  ^ nll"\l+C  ^ "Bp+l^nll"^ll  ^  ^  } 


m 


P(P+1)  Bp+l(ni1  5n+l)  +  (5^+C^)1 


f  ,  -(mfl)  -(m+l)  . 

+  0  {max  ((a^+I^)  ,  (a^-K^)  )  j  . 


Ucing  the  identity  (see  e.g.  [1] ) 


n 


Bn(x  +  h)  -  'l  §)\M  h 


n-k 


koO 


(3-16) 
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log  BostO^)  oc  (1^+^+  |)  log(ni;L+R1)+(n11+C1+  Ijlog^-K^) 


-(N+an+r+c-  |)log  (N+ti^)  -  log  rCn^+l)  + 


m  p+1 


v  v  '-1)P(^1) 

+  I  1  -  -  Vvaii)x 


p=l  k«*0 


P  (P+1) 


rfM_i _ 

ul  xP  1  t*  t  n  \P  f-  j-n  \P' 


(3.17) 


(N +0^?  '  (1^11+R1)P  (^1): 


r  /  -(m+l)  -(nrt-lX-v 

+  0  |max  (v(\1+R1)  ,  (^ii+Ci)  j)  > 


where 


% 


{ 


1,  k  o  0,  lj • • • ,P 
k  o  p  +  1. 


Hence 


“n  "n 


(n^+IL)  (^n+c1)  (N+n^  ) 

Post  (n. .  )  a - = - +  remainder  which  goes  to 

n....  !  zero  as  the  observed 


‘11  • 


frequencies  -*  ®  . 


(3.18) 
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Using  the  relationships  given  by  ( 3  •  10 )  we  thuB  have  that  the  large 


sample  posterior  distribution  of  n. .  is  the  Poisson  distribution  with 

N  Mi 

mean  n^  ■  -  ,  i.e. , 


l-vA 


-  N  *A\r « 


Post  (nu) 


r- "  " 

exP\- 


a. 


0,  1,2,.. 


n 
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